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Evaluation 
Bibliography 



This annotated bibliography is intended to provide selected references 
which complement or supplement the Evaluation Checklist , the Workbook on 
P rogram Evaluation , and the Evaluation Guide . This bibliography does not 
purport to be exhaustive but. rather, serves to suggest alternative resour 
Included in this document are entries under the following headings: 

• Evaluation of Innovative Practices 

• Checklists 

• Formative or Process Evaluation 

• General Evaluation Issues 
. • Methodological Issues 



^VALUATION 0? INNOVATIVE PRACTICES. 

1. Duiin, J. A. Evflliia*t:lon- St udied of the AIR Career gdiicatlon 
Curriculum and Curriculum Products « Volume 5. Palo AI^Pi 
Calif.: American Institutes for Research in the Behavioral 
Sciences, 1974. (ERIC No. ED IQl 207) , 

This document details the total project evaluation efforts 
of the AIR Career Education Curriculum Development Project for 
grades K-9. The plan for Implementation, dlsaemination, 
evaluation and utilization activities Includes: 1) an annotated 
bibliography; 2) an objectives catalog; 3) a curriculum guide; 
and 4) a set of 14 sample^ Instructional units. 

2. Freeman, R. E. Curriculum Materials Zvaluation as r Process 
for C hanging Educatloni Wo rk of the Diablo ValleV MWflrtOa 
Project . Orlnda, Calif.: Diablo Valley Education Project, 1974. 
(ERIC No. ED 099 283) 

The objective of the program was to use materials evaluation 
as a means to make local schools effective instrunenta in teaching 
about human dignity and global problems. The program was designed 
to use the existing school structure, Involve the community, ana 
set up a self-evaluation to test the results. Materials of 
evaluation were chosen according to global perspective, organization 
of content, quality of supplementary materials, flexibility, format, 
reading level, and cost. 



3. Lundln, S. C, and French, R. The Application of Anthropological 
Experimental Schools Evaluation . Washington, D.C.: National Institute 
of Education (DHEW) , 1973. (iftlC No. ED 087 416) 

General descriptive material on the Experimental Schools Program 
followed by specific discussions of the two major components of the 
evaluation— the Anthropological Study (a case study approach designed • 
to describe the evolution of the program) and the Impact Study (designed 
to assess the effect of the program upon the schools, the students and 
their parents, and the community) are presented. The assessment stratep. 
data to be collected, methodological considerations, dependability of the 
data, and the analyses required are discussed for each compenent. 
Limitations of the evaluative strategies are also discussed. 



Athens, Ga.Mtolversity of Gepjrgia* 196$^ (fSRXQ^ «^^ ED 045 51?) 

Bevlawed m this suansfy ari six of tha saveti evaluatlotjs 
completed by partlcipaats i^ the Anthropology CWiculu* Project (ACP). 
These seven are; I) cognitive .achievement within the premises of 
a single discipline approach and differ6ntiaX te*6her prepAtatlon; 

2) difierential cognitive aehievenent by grade level vith treatment 
.by conventional elementary metho4s and by progranmed instruction; 

3) different cognitive achievement varying treatment by deductive 
and inductive methods of teaching and by relationship to teaching 
style as perceived by teachers and obiervers; A) oognitiVe achievement 
for five-year olds using an adaptive oral unit; 5) pupil iudgemente of 
interest suitability; and 6) processes of currlculwii diffusion. The 
bibliography cites sources for complete descriptions of these evaluations. 

5. Witts, R. Improving Curriculum— Focus on Mathematics, yi^rust for 
Educational Leadership . 7, A.. 1978. 

This document reports the evaluation of two ESEA Title HI elementary 
math projects. A set of criteria prepared to help in making a * first 
evaluatloa'' of any similar curricula is included. 

■ «•:. • 

CHECKLISTS 

- 6. Altschuld, J. W., and Baker, D. Project 4-E t iivaluatlng Elementary 
Education Effectively . (ERIC No. ED 059 259) 

This elementary school evaluation checklist covers organisation and 
administration, curriculum and Instruction, pupil services, staff personnel. 
Instructional materials and equipiient, school/ community relations evaluation 
and reseach, and physical facilities. 

7. _^ A Suggested Checklist for Assessing a Sc ience Program. 

(Report No. OE 29034-A) Washington, D.C.: Of fie of Education (DREW). 
(ERIC No. ED 137 55) 

This document contains a checklist for the evaluation of elementary 
and secondary school , science programs. An introductory section deals with 
the importance of timely evaluation metljods and of broad-based participation 
Explanations for the.^evelopment and use^of a program profile and the use of 
a checklist for evaluat4e»-are given. 
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8. fti^^vwn. fl. A Checklist «^''^^»^i**fl; Bl*«antarv Sodml Stu41fla 
Materials , Rosevllle, Minn. 8 Social Studies Setvlce Center. 197 A. 
(ERIC No. ED 090 132) 

A scale, summary sheet, and checklist provide a guide for aelectlng 
elementary social studies materials. The checklist, based o6^ trends In 
social studies as found In professional literature and pubXijhed social 
studies materials, is appropriate for use by Individual tjachere, 
curriculum committees and curriculum coordinators. The checkllet contains 
an overview of materials, textbook series, and the conceptual structure of 
the materials; sections concerning cultural e<ruallty» topllolt biases, 
self -development and values; Information on format, modes or Instruction 
and evaluation; and ff. statement of reviewer's general Impressions. 
Instructions for the checklist are followed by a scale for approadiaatliig 
the reviewer's perceptions of materials as related to the specif ic checklist 
items. Sources used In the construction of the checklist are Included. 




This checklist is designed to assist school system personnel In 
evaluating the media programs of their schools. Listed are criteria 
against which each of the central components of the media progrwft can 
be measured. These components are: 1) commitment on the part of staff^ 
members to media use; 2) the inclusion of media services as "J^i^^egral 
part of curriculum and instruction; 3) the adequacy of the media center s 
functioning: 4) the suitability of the physical facilities "^^J^^ 
program; 5) the availability of adequate financing; and 6) the allocation 
of sufficient staff to the media program. 
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10. Lasher, E. B. Evaluative Criteria of non-print materials. Audiovisual 
Instruction , 20, A» 16-17, 1975. 

This article offers a checklist for the evaluation of non-print 
materials. 
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TOM ^TIVE OR P»0C«8S EVALtyATIOW 



11. AataBore,.W. H., Toward an tv^loAtion of 'lnAi^t^^^ 
Hadlaon, Wis.; wuconsln SUte Dep«rtn«nt 0^ ^hlXt ittptji^cij|tPtt»..,W.T5» 

(Basic No. BD no 508) . • . ^ M^^^fe 

Although developed as an alternative approach to fvalui^^ijjt. thd 
extent to yhlch the model for Ind^idually Qvilded Edut^t^on (xee) 
been Implemented, the evaluation Hitrustsnt desctlbed ilU^y liisad 
variety of Innovative curricula. Tb3 type of asa«*ii^i|^^ 
provides Improvement-oriented Information to deCltlO|^j?iiJc^ 
them to document their Implementation of th6 IGE syatiiiil ^^ " 




program areas where modifications and lmprovene&]t0^<|i^js 



Contlnuous_Currlc^^ 



GoshenT^nnanar^irlleldO 

Program development and evaluation procedures Involvliag faculty* 
students, and community In a continuous curriculum development project are 
given In this second'^year report of the Fairfield, Indiaiui Contounity School 
The report Is divided Into six sections: 1) the process of developing 
conceptually designed curricula; 2) the processes of the professidnal staff; 
3) analysis of professional growth; A) analysis of the administrator/ teacher 
relationships; 5) dissemination process and cost; and 6> the most algnif leant 
change during the second project year. Exhibits illustrating workshop 
actlvltes, as well as statistical evaluations of the project's impact on 
teachers and stiidents, are appended. * 



13. Hall, G. E., and Loucks, S. F. A Developmental Model for Determining 
Whether the Treatment la Actually Implemented. American Educational 
Re/ earch Journal . 14, 3, 263-76, 1977. 

An important part of evaluation la to determine whether the "treatment" 
is being Implemented as designed. Hall and Loucks describe "levels of use 
of the innovation" as a concept which yields an operational, cost-feasible 
description, and docimientation of whether or not an educational innovation 
or treatment is being implemented. 
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14. Kaskowlez, D., And Stalllngs, J. An Aaaasflnent Qjg groaran 
Implementation 1^ Project Follow Throuiih . Menlo Park, Calif; s 
Stanford Research Institute, 1975. (ERIC No^ BD 106 349). 



Methodological Issues and results described in this paper 
originated from a Stanford Research Institute evaluation of 
classroom observation data collected in the spring of 1973. The 
main question addressed in this evaluation was whether each of seven 
Follow Through sponsors had successfully implemented the program in A 
variety of sites. The steps in the evaluation of implementation included: 
1) a determination of the essential program components; 2) « translation 
of the components in terms of observable phenomena; 3) a measure of the 
phenomena; and A) a standard by which to Judge implementation. These steps 
provide a useful guide for the evaluation of innovations In genaral and are 
not restricted to Follow Through Programs.. 



■ ■ ■■ -'fymm 



15. Novicks, S. The Use of Formative Evaluation Procedures for Improvement 
of a Socially-oriented Course in Chemistry, Studies in Educational Evaluation, 
2, 1, 1-7, 1976. 

Unstructured and structu|ed evaluation modes were used to probe teacher 
perceptions of a socially-oriented chemistry course for non-science majors. 
Students' assessments reinfo.'ced teacher percepclons. This procedure yielded 
information concerning course content, societal relevance, motivation. 
Interest, laboratory and demonstration exercises, other classroom or 
individual exercises, and teaching problems. 



16. . Pilot Guidelines for Improving Ins4a?u ctlonal Materials 

through the Process of L( arner Verification a nd Revision. 

New York: Educational Products Information Exchange Institute, 1975. 

(ERIC No. ED 112 822) ^ 

> 

An ongoing effort for the improvement of instructional materials 
based on syetematic feedback from learners who have used the materials, 
this evaluation provides a method for identifying instructional strengths 
and weaknesses of a curriculum. Reporting and assessing LVR activities 
are carried out according to a flexible format. The format includes 
a) descriptive information on the product; b) instructional design;, 
c) intended learner outcomes to be investigated; d) conditions of use 
of products; e) techniques for gathering feedback; f) descriptions of 
learners used in LVR process; g) analyses of findings; h) specific 
improvements made; 1) background and fufure of products. Three appendices 
and a review of Task Force reactions are included. 
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17. Shanti, M. The teamer Verification of Series R: The New >tocMlllan - 
Reading Program. Highlights * Amherst, Mass.: National Evaluation Systems, 
Inc. 1975. 

This document Includes a summary of the leamer-verlflcatlon approach 
to evaluation and a report about the application of the procedure to the 
"Series R" reading program developed by the liacMlllan Publishing Company. 
The report describes five components of learner verification (overall 
student achievement, specific student achievement^ specific student growth 
and retention, and student and teacher opinion and satisfaction data), 
discusses the evaluative^ findings for each of these components, summarizes 
the overall evaluation, and contains charts illustrating the findings. 



18. Salmon, S., and Glassbi^.rg, S. Behavioral Evaluation as Means of 
Analyzing Student Progress in Psychological Education. School Counselor , 
23, 2, 91-98, 1975. 

Two procedures are described that are used in psychological education 
and aimed at helping students identify and internalize cognitive and 
effective changes. The first method is the setting of weekly behavioral 
objectives. The second is a behavioral evaluation completed at the end of 
each semester by the students* 

19. Wittrock, M. C, and Wiley, D. E. (Eds.) The Evaluation of Instruction 
Issues and Problems . New York: Holt, Rinehart and Winston, 1970. 

This reference provides presentations on general evali^tion issues 
in the areas of curriculum planning, design, and revision. Glaser's 
article on "Evaluation Models" is particularly of interest in the broader 
perspective of evaluating innovative practices in education. Major topics 
are evaluation theory. Instructional variables, contextual variables, 
criterion variables, and methodological issues. Based on a symposixmi, the 
format of this text includes stimulus papers followed by structured 
comments and follov-up discussions. 

20. Walberg, H. J Evaluating Educational Performance . Berkeley, Calif.: 
McCutchan, 1974. 

Described as a sourcebook of methods , instrtmients , and examples , 
this text contains a wide variety of articles touching on such disparate 
topics as teacher effectiveness, needs assessment, learning environments 
and trend-surface analysis. 
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?l. Tyler, R. W, (Ed.) Educational Evaluation; New' R oles. New Means. 
Chicago: University of Chicago, 1969. 

< 

This collection of articles presents a comprehensive description 
of educational evaluation practices at the end of the 1960* s. The 
contributors to this text discuss significant issues a till relevant 
today Including the role of evaluation in guidance, admissions, 
classification, and selection. Cross-cultural evaluation methodologies 
are also discussed. 

22. Suchman, E. A. Evaluative Research . New York: Russell Sage 
Foundation, 1967. 

This text presents Important issues In the broad area of program 
evaluation. Included are chapters on evaluation principles, design, 
and administration. Many difficult evaluation problems are discussed 
in the context of social experimentation and evaluative research. 

23. Struenlng, E. L.. and Guttentag, M. (Eds.) Handbook of Evaluatlo_n 
Research . (2 vols.). Beverly Hills, Calif.: Sage Publications, 1975. 

Thirty-seven articles on a variety of topics are included in these 
two volumes. Included are such topics as politics and values in 
evaluation research, cost-benefit analysee and design of evaluation 
suudles. Also included are special topic papers on early intervention 
evaluation of public health programs and new careers progrfims. 

24. Scrlven, M. Goal-free Evaluation. In R. E. House (Ed.), School 
Evaluation;' The Politics and Process . Berkeley, Calif.: McCutchan, 
1973. 

Scrlven' s concept of goal-free evaluation is presented here. 
Goal-free differs from traditional evaluation lu that the evaluator 
does not know whatthe program objectives are. Thus, this type of 
evaluation depends on the acute observations and to some extent, 
intuitions of the "blind" evaluator. 



9 



ERIC 



8.^ 



25. Scriven, M. The Evaluation of Educational Goals* Instructional 
Procedures and Outcomes . 1972. (ERIC No, ED 079 394) , 

A model checklist conceptualizing the evaluation process is 
presented and discussed. It la quite general and is Intended to apply 
to the evaluation of educational products, procedures, and most outcomes. 
The Pathway Comparison Model presented consists of the following: 
1) characterisation — how generally or specifically to describe the 
"treatment"; 2) clarification of conclusion with client-award of merit, 
best buy, etc; 3) causation; 4) a comprehensive check of consequences; 
5) conceptualization; 6) costs; 7) consumer characteristics; 8) critical 
competitors; 9) credentialing; and 10) concluislons and communications. A 
detailed checklist for product evaluation is appended. 

26. Rose, C, andNyre, E. G. The Practice of Evaluation. Princeton, 
N. J.: ERIC Clearinghouse on Tests, Measurement, and Evaluation, 1977. 

This monograph is divided into two sections. The first section is 
an overview of theoretical concerns in program evaluation. The eecond 
Section contains descriptions of evaluations. A comprehensive bibliography 
is attached. 



27. Popham, W. J. Educational Evaluation . Englewood Cliffs, N. J.: 
Prentice-Hall, Inc., 1975. 

This collection Includes articles on evaluation perspectives, 
formative and summatlve evaluation, data analysis and reporting, 
criterion-referenced measurement, cost analysis, and matrix samplings 
Each article was produced to stand independently; and, as a result, there 
are contradictory viewpoints expressed. 



28. Stufflebeam, D. L., Foley, W. J., Gephart, W. J., Cuba, E. G., 
Hammond, R. L., Merriraan, H. 0., and Provus, M. M. Educational 
Evaluat:'.on and Decision-Making. Itasca, III.: Peacock Press, 1971. 

This text presents a comprehensive overview of evaluation as a 
concept and of evaluation models commonly found in the educational 
literature. Also Included are discussions of meta-evaluation, evaluation 
methodology, organization of evaluation tuilts and the training of . 
evaluators. 



10 



29. Patton, M. Q. Utilization Focuaed Eyaluatlon . Beverly Hills, 
Calif.; Sage Publications, 197«. 

This tsixt Integrates the political and soclolo|i;l6^1 aspects of 
program evaluation. In so doing, Patton reminds the reader that an 
Important feature of evaluation is that it will not be «iaed unless 
it is useful. He suggests that the use is designed by the potential 
user of the information and not by the evaluator.^ Thus, in order to 
have evaluation results ^sed', the evaluator must answer questions 
posed by individuals who receive the evaluation results. 



30. Knoepfner, R. Report on the Plel<? Testing of the CSE Elementary 
School Evaluation Kit: Needs Assessment . Los Angeles, Calif.: Center 
for the Study of Evaluation, 1971. (ERIC No. ED 058 673) 

This evaluation kit is a self-instructional package that provides 
a principal with all the procedures and materials necessary to conduct 
an assessment of his/her school's educational needs. This report 
describes the field testing of the kit during the 1970-71 school year. 
Conclusions made on the basis of the fi-eld tasting and recommendations 
for changes in the kit prior to its final publication are presented. 



GENERAL EVALUATION ISSUES 

31 . Ab t , C . C . (Ed . ) The Evaluation of Social Programs . Beverly Hills , 
Calif. J Sage Publication, 1976. 

This collection of articles focuses on issues in the evaluation of 
social programs. Included as major topics are social experiments, policy 
impacts, evaluation payoffs, research versus evaluation, and evaluation 
of educational programs. 

32. Anderson, S. B. , and Ball, S. The Profession and Practice of Program 
Evaluation . San Francisco, Calif.: Jossey-Bass, 1978. 

The authors describe this text as a workbook which includes simulated 
materials for planning and implementing evaluations. Major topics 
discussed are evaluation practices, ethics and values in evaluation, and 
che future of program evaluation. 
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33 Anderson, S. B.. Ball', S., Murphy. R. T., and Associates. 
p';v.Y^nSia of F^nn..lonal Evaluatio n. San Francisco . Galif . s 
Jossey-Basa, 1975. 

m response to the increasing demands for evaluation of federal, 
r^alonal and local programs, this encyclopedia is an attempt to bring 
ordtr S'the fleH. LcludeLare major techniques and concepts described 
in Sr^s tSat are intended tKe "generally comprehensible to program 
idmi;"tra?ors. funding agents, and students n^^^^.f 
well as to the social scientists and measurement specialists who have 
tended to dominate it" (p. VIII). The articles are arranged 
Suf are also listed under major topics in the Table of Contents. Major 
topics discussed include Evaluation Models. Functions, Design, and 
Measurement Considerations. 

34. Blackwood, P. E., and Porter, T. R. How to Evaluate Scie nce Learning 
in the Elementary School . (Report No. 471-14564). Washington D.C. : 
National Science Teachers Association, 1968. (^IC No. ED 027 179) , 

Discussed are generalizations, criteria, and techniques for student 
and teacher evaluation in elementary school science. Part 1 relates 
examples of evaluative activities in first, fifth, and sixth grades and 
briefly discusses them relative to: 1) the relationship between 
evaluative activities and the objectives of the science lessons and 2) 
the purpose of the evaluation. Part 2 emphasizes that the purpose for 
science teaching must be clear ajad d-'.scusses three objectives commonly 
accepted for elementary science programs. Part 3 discusses the use of 
behavioral objectives as an avenue for evaluating the objectives discussed 
in Part 2, Various examples of instruments for evaluating student growth 
are included. A self -evaluation checklist for teachers is also provided. 

35. Bloom. B. S., blastings, J. T., and Madaus G. P. Handbook of Formative 
and Summattve Evaluation of Student Learning . New York: McGraw-Hill Book 
Company, 1971. 

This text is designed for those interested in constructing cognitive 
and affective tests for classroom use. The first part of the text 
describes formative and suramative evaluation strategies and general test 
construction practices. The second part of the text details the evaluation 
of instruction in specific content areas. 
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36 n.,..4:oo/M: Rlenantarv Sch»Ql Social SM dleat A Guide tO Current 
Rt;eardn Wasiiington" d!c.: Aaeociatlon for Supervision and Curriculum 
Development, 1970. (ERIC No. ED 043 552) 

Section 6 of this document summarizes evaluation Practices In the 
area of elementary school socla^ studies. This summary Includes research 
and theory regarding promising techniques, expert approaches, and 
curilculum evaluation. 

METHODOLOGICAL ISSUES \^ 

37. Bloom. B. S. TAvonomv of E d ucational Ohlectlves: Cognitive Domain . 
New York: David McKay, 1956. 

This text provides one method for organizing a learning hierarchy in 
a subject area. This may be used in specifying objectives, designing tests, 
and organizing and evaluating cognitive instruction. 

38. Campbell D. T. Reforms as experiments. American Psychologist, 
24, 4, 409-429, 1969. 

This is a discussion of the dilemma faced by evaluators attempting to 
adhere to sound research practice while dealing In a humane fashion with 
thr^rticipants in the evaluation study. Although specifically focused 
on s^lal service programs, the issues raised here are Important In the 
design of educational programs as well . 

» 

39 Campbell, D. T., and Stanley, J. C. Experimental and Quasi-experimental 
D!;ignrfor R;search on Teaching. In N. L. Gage, (Ed.), Handbook of Research 
on Teaching . Chicago: Rand McNally, 1963. 

— — 

Campbell and Stanley review selected experimental designs and a 
variety of quasi-experimental designs. Included are l^^Pi^^^'^J^'*"/"!^!^^^ 
and problems. A concise presentation of the threats to internal and external 
validity is included in the discussion of each design. 

40 ampbell, D. T., and Erlebacher, A. How Regression Artiflcants in 
^;si:^erim;nt;i Evaluation Can Mistakenly Make^Compensatory Education 
Sok HaS?^. In J. Hellmuth (Ed.), Dlsadvantased Child. New York: 
Brunner/Mazel, 3, 185-210, 1970. 

Stimulated by the evaluations of Project Head Start, these authors 
h;?ve described various sources of bias common in evaluations of compensatory 
education. Of particular interest is the problem of identifying an 
appropriate comparison group. 
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Al". Gronlund, N. E. Preparing Crltelon-ref e renced Tests for qiassroom 
Instruction . New York: The^MacMlllan Company,, 1973. ^ 

Gronlund has condensed Instructions for the development of teacher- 
made criterion-referenced teats In this document. Included are careful 
explications of each step of achievement test construction from the 
specification of objectives to simple item analysis procedures and the 
Interpretation of test results. 

42. Weiss, C. H. Evaluation Research; M p<-^QdB of Assessing Program 
Effectiveness . Englewood Cliffs, N. J.: Prentice-Hall Inc, 1972. 

Weiss presents adaptations that can be used to alleviate the problems 
faced by evaluators who try to implement experimental designs in real world 
situations. In particular, suggestions on how to deal with comparison or 
control groups are discussed. 

43. Krathwohl, D. R. , Bloom, B. S., and Masia, B. B. Taxonomy of 
Educational Oblectives; Affective Domain . New York: David McKay, 1964. 

This text is complimentary to the Taxonomy of Educational Oblectivest 
Cognitive Domain and provides one method for organizing an affective 
hierarchy. This text may be used in specif yinfl objectives, designing 
instruments and in organizing and evaluating affective learning. 



44. Sanders, J. R. ; and Natzlger, D. H. A Basis for Determining the 
Adequacy of Evaluation Design . Portland, Ore.: Northwest Regional 
Educational Lab, 1975. (ERIC No. ED 127 345) 

Procedures are suggested to determine the adequacy of evaluation 
'designs prior to actually conducting the evaluations. First, basic 
questions are posed, such as, "Why evaluate?" Second, a checklist 
of basic considerations in Judging evaluation design is presented. Third, 
a sample ^ign if presorted. Fourth, the reactions of professional eval- 
uators to the adequacy of evaluation designs are included. 



4 

u 



13. 



A5. Scrlven, M. Evaluation Perspectives and Procedures. In W. J; J^P^f 
(Ed.), Evaluation In Education: Current Applications. Berkeley, v9ll£.. 
McCutchan, 1974. 

This article on evaluation perspectives and procedures Is divided 
Into six sections. The first section briefly discusses qualitative and 
quantitative research and evaluation. In the second section there is an 
exploration of the utility a^ validity of a checkltst-thaE^can be used 
to evaluate products; as an instrument for evaluating prodijcers; for 
weighting evaluation proposals and production proposals; and as an 
instrument for evaluating evaluators of products, producers, etc., since 
it is asserted that competent evaluation must cover each o^^hese. points . 
The third section discusses prospects and problems In ^""^IZ^^l^ 
(cm which the author sees as the evaluation of actual effects against 
(typically) a profile of demonstrated needs In this area of education. 
The section that follows contains comments by Stufflebeam, Alktn, Popham, 
Ind Kneller, with replies by the author. In the fifth section Procedures 
are outlined representing a set of caudal ^^^f "^^^f ^ 

operandi ^0) analysis is probably the most distinctive. Full conversion 
of the MO method into quantitative techniques may or may not be possible, 
but some suggestions as to procedure are listed. In section six, cost 
analysis in evaluation and the doctrine of cost-free evaluation are 
discussed in an effort to lay the ground for others to provide answers. 

46. Scrlven, M. The Methodology of Evaluation. In R. E. Stake, (Ed.), 
P^rappotlves of Curriculum Evaluation . (AERA Monograph series on 
Curriculum Evaluation, I) Chicago: Rand McNally, 1967. 

in this monograph, Scrlven points out that conceptions of evaluation 
are inadequate in both philosophical and practical terms. He then offers 
the distinction between evaluation goals versus roles, formative versus 
summative evaluation, prof essiotial versus amateur evaluation, and 
evaluation studies versus process studies as foci for Improving the basis 
for program evaluation. 

47. Smith, M. S., and Bissell. J. S. Report analysis: The Impact of Head 
Start. Harvard Educational Review , 40, 1., 51-104, 1970. 

An Important part of the histroy of modern educational evaluation, 
the initial evaluation of Project Head Start raised Important methodological 
Issues which are pointed out in this paper. In particular, the problem 
of aggregating the data from various Head Start Centers is discussed. 
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These materials were developed i>y Educational Testing Service 
- Southern Regional Of f ice , Project Director Margaret B. Weber, to be 
used in conjunction with the Evaluation Improvement Program, Workbook 
on Program Evaluation . 
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